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The amount of literature on program evaluation decreased in 



1996, continuing a trend begun in the late 1980s.. One exception to this is 
the literature on assessment. Another frequent issue is the technique of 
evaluation. Many examples of research on evaluation are from international 
settings, where accountability and evaluation appear to be integrated into 
institutional systems. Student learning and satisfaction with services tend 
to be the focus of evaluation and assessment . An exception to the ‘trend away 
from systemic evaluation is a paper by Aquirre and Hawkins, "Why Reinvent the 
Wheel? Let's Adapt Our Institutional Assessment Model," which describes an 
integrated assessment and strategic planning process designed to comply with 
accountability requirements at a community college in New Mexico. Most 
evaluation studies occur as students leave school or after they have 
graduated, and it would be helpful to have more evaluations while students 
were still in school . Some articles suggest ways to improve the process of 
evaluation. One area where research on evaluatior. is prevalent is research on 
new or nontraditional academic experiences, such as first -year seminars or 
bridge programs, experiential learning, and distance education. Some 
evaluation focuses specifically on faculty members, exploring the quality of 
teaching and research. Research on staff members' impact on student learning 
or experience is minimal, (Contains 12 references.) (JM) 
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Program Evaluation 

by Adrianna J Kezar 

The amount of literature on program evaluation is down from previous years, continuing a 
trend begun in the late 1980s. Although the research on program evaluation is declining, 
that on assessment is on the rise. Approach or technique of evaluation is a frequent issue, 
including which groups (students, faculty, staff), time periods (before, during, or after 
college), or level (classroom, department, school, college) should be studied. Many 
examples of research on evaluation are from international settings, where accountability and 
evaluation appear to be more integrated into institutional systems (Brennan et al). Students' 
learning and satisfaction with services tend to be the focus of evaluation and assessment. 

Approaches to Evaluation 

One of the most notable trends is that systemic or comprehensive evaluation is not 
widespread in the literature, the research, or programs in practice. One exception to this 
trend is a paper by Francisco Aguirre and Linda Hawkins, "Why Reinvent the Wlieel? Let’s 
Adapt Our Institutional Assessment Model," which describes an integrated assessment and 
strategic planning process to comply with requirements for accountability at the community 
college affiliated with New Mexico State University at Alamogordo. The article describes 
assessment tools that include surveys from the community, students, faculty, and 
employers, a data form from new students, a survey of withdrawing students, and a high 
school survey. 

Most evaluation occurs as students leave school, some occurs after students have graduated 
(Webb et al.), and occasionally research or evaluation is performed while students are still 
in school (Donald and Denison; Kowlowitz et al; Rowlands et al.). For example, Donald 
and Denison present a Canadian institution's experience with assessing graduates' 
undergraduate experiences, examining the extent to which broad indicators of performance 
could be used to improve university programs. If programs are being evaluated, the results 
are not being shared or published. This outcome could be the result of the publication 
process, which prefers original research or new ideas, not the evaluation of current 
programs. A Web site sharing different types of program evaluations would be an asset, 
providing those interested in developing evaluations with a point of comparison for their 
results. More long-term and longitudinal studies are needed, and it would be helpful to have 
more evaluation while students are still in school, not just as they are leaving or after they 
are gone. 

Some articles about evaluating research suggest ways to improve the process of evaluation. 
For example, John Biggs argues that institutions implicitly encourage quantitative 
assessment, thus encouraging a surface approach to learning, although qualitative methods 
support a deeper approach and should be integrated into the assessments. The research on 
methods for evaluating programs, departments, or institutions is very important. Institutions 
are struggling to develop successful evaluations that give them useful information, not just 
information that is placed on a shelf to collect dust. 
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Evaluating New Programs and Innovations 

One area where research on evaluation is more prevalent is new or nontraditional academic 
experiences, including first-year seminars or bridge programs, experiential learning, and 
distance education (Hanley and Olson; Reid et al). For example, one published evaluation 
of a precollege program identifies several key elements in a program's success: course 
content, teaching techniques, and support for personal, psychological, and altitudinal 
changes (Hanley and Olson). An evaluation of experiential learning examines the validity 
of supervisors’ assessments; validity appears to have been enhanced by students' use of a 
well-specified intervention model and structured recording forms. Maty Hayden and Jeff 
Thompson describe an evaluation of distance learning from overseas. The results suggest 
that the instrument is useful not only in determining satisfaction, but also in evaluating the 
relative contribution of specific program components to a program's overall success. 

Evaluating Teaching and Research 

Some evaluation focuses specifically on faculty members and explores the quality of 
teaching and research (see also "Faculty"). An accompanying body of research evaluates 
faculty research. Although cun*ently limited, it will likely become an area for research, 
given public pressures for accountability, quality teaching, and faculty productivity (Florida 
State Postsecondary Education Planning Commission). William Patrick and Elizabeth 
Stanley, in "Assessment of Research Quality," describe the British experience with quality 
assessment of research and newly introduced measures to improve research programs' 
accountability. 

Research on staff members' impact on students' learning or experiences is minimal. Some 
institutional studies focused on service examine this aspect, but few national studies look at 
' the impact of housing staff, counseling, financial aid, and other campus services on 
students. This area clearly needs research. Various functional units’ methods of evaluation 
and results could be posted on a Web site to help provide a model for institutions and 
programs. 
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